API Docs¶

Production::
https://kcmj21es3e.execute-api.us-east-1.amazonaws.com/prod/v1/docs
https://kcmj21es3e.execute-api.us-east-1.amazonaws.com/prod/v1/redoc

This Page Source Code:
https://github.com/Radio-Memory/radiomemory-ai-api-demo

In [ ]:
import config as cfg
import requests
import cv2
from PIL import Image
import io
import base64
import matplotlib.pyplot as plt
import pprint

pp = pprint.PrettyPrinter(indent=4)

BASE_URL = "https://kcmj21es3e.execute-api.us-east-1.amazonaws.com/prod/v1"

def encode_image(image):

    buffered = io.BytesIO()
    image.save(buffered, format="JPEG")
    image64 = base64.b64encode(buffered.getvalue())
    return image64.decode()

Authentication¶

In [ ]:
response = requests.post(
        BASE_URL + "/auth/token",
        data={
            "username": cfg.USERNAME,
            "password": cfg.PASSWORD,
        },
    )
token = response.json()["access_token"]
headers = {"Authorization": "Bearer %s" % token}

Panoramics¶

In [ ]:
# The image is converted to grayscale in the backend, so you can convert it before sending in order to save bandwidth.
panoramic_image = Image.open("images/panoramic0.jpg").convert("L")

Dentition Endpoint¶

Classify the panoramic radiography in either toothless, superior toothless, inferior toothless, mixed or "normal".

In [ ]:
dentition_response = requests.post(
        BASE_URL + "/panoramics/dentition",
        json={
            "base64_image": encode_image(panoramic_image),
        },
        headers=headers,
    )
In [ ]:
pdata = dentition_response.json()
pdata["entities"] = pdata["entities"][:4]
pp.pprint(pdata)
{   'entities': [   {   'bbox': None,
                        'class_name': 'normal',
                        'contour': None,
                        'line': None,
                        'point': None,
                        'score': 1.0,
                        'tooth': None,
                        'type': 'dentition'},
                    {   'bbox': None,
                        'class_name': 'superior_toothless',
                        'contour': None,
                        'line': None,
                        'point': None,
                        'score': 7.882013930426979e-10,
                        'tooth': None,
                        'type': 'dentition'},
                    {   'bbox': None,
                        'class_name': 'mixed',
                        'contour': None,
                        'line': None,
                        'point': None,
                        'score': 2.5856181778971532e-11,
                        'tooth': None,
                        'type': 'dentition'},
                    {   'bbox': None,
                        'class_name': 'inferior_toothless',
                        'contour': None,
                        'line': None,
                        'point': None,
                        'score': 1.471757177934352e-12,
                        'tooth': None,
                        'type': 'dentition'}],
    'height': 1292,
    'image_hash': None,
    'width': 2444}
In [ ]:
fig, ax = plt.subplots(figsize=(10, 10))
ax.imshow(panoramic_image, cmap="gray")
ax.axis("off");
No description has been provided for this image

Longaxis Endpoint¶

Returns a point coordinates for the crown and the root of every tooth in the panoramic image.

In [ ]:
longaxis_response = requests.post(
        BASE_URL + "/panoramics/longaxis",
        json={
            "base64_image": encode_image(panoramic_image),
        },
        headers=headers,
    )

Output Sample¶

In [ ]:
pdata = longaxis_response.json()
pdata["entities"] = pdata["entities"][:3]
pp.pprint(pdata)
{   'entities': [   {   'bbox': None,
                        'class_name': '18_0',
                        'contour': None,
                        'line': None,
                        'point': [632.8214285714287, 553.7142857142857],
                        'score': 0.5159812122583389,
                        'tooth': None,
                        'type': 'longaxis'},
                    {   'bbox': None,
                        'class_name': '18_1',
                        'contour': None,
                        'line': None,
                        'point': [611.0, 351.83928571428567],
                        'score': 0.5159812122583389,
                        'tooth': None,
                        'type': 'longaxis'},
                    {   'bbox': None,
                        'class_name': '17_0',
                        'contour': None,
                        'line': None,
                        'point': [752.8392857142858, 594.0892857142857],
                        'score': 0.594242125749588,
                        'tooth': None,
                        'type': 'longaxis'}],
    'height': 1292,
    'image_hash': None,
    'width': 2444}

Visualization¶

In [ ]:
from vis import draw_longaxis_output

dimage = draw_longaxis_output(panoramic_image, longaxis_response.json()["entities"], draw_axis=True)
fig, ax = plt.subplots(figsize=(10, 10))
ax.imshow(dimage)
ax.axis("off");
No description has been provided for this image

Panorogram Endpoint¶

Returns the panorogram curves of the panoramic image.

In [ ]:
panorogram_response = requests.post(
        BASE_URL + "/panoramics/panorogram",
        json={
            "base64_image": encode_image(panoramic_image),
        },
        headers=headers,
    )

Output Sample¶

In [ ]:
pdata = panorogram_response.json()
pdata["entities"] = [{**p, "contour": p["contour"][:3]} for p in pdata["entities"][:2]]
pp.pprint(pdata)
{   'entities': [   {   'bbox': None,
                        'class_name': 'ContMand',
                        'contour': [[150, 179], [149, 180], [148, 180]],
                        'line': None,
                        'point': None,
                        'score': None,
                        'tooth': None,
                        'type': 'panorogram'},
                    {   'bbox': None,
                        'class_name': 'CanManDir',
                        'contour': [[386, 547], [385, 548], [384, 548]],
                        'line': None,
                        'point': None,
                        'score': None,
                        'tooth': None,
                        'type': 'panorogram'}],
    'height': 1292,
    'image_hash': None,
    'width': 2444}

Visualization¶

In [ ]:
from vis import draw_panorogram
dimage = draw_panorogram(panoramic_image, [[p["class_name"], p["contour"]] for p in panorogram_response.json()["entities"]])
fig, ax = plt.subplots(figsize=(10, 10))
ax.imshow(dimage)
ax.axis("off");
No description has been provided for this image

Metals Endpoint¶

Returns metal structures found on the panoramic image.

In [ ]:
metals_response = requests.post(
        BASE_URL + "/panoramics/metals",
        json={
            "base64_image": encode_image(panoramic_image),
        },
        headers=headers,
    )

Sample Output¶

In [ ]:
pp.pprint(metals_response.json())
{   'entities': [   {   'bbox': [   860.0716552734375,
                                    624.821044921875,
                                    1569.6455078125,
                                    783.9153442382812],
                        'class_name': 'ortodontia',
                        'contour': None,
                        'line': None,
                        'point': None,
                        'score': 0.9565339088439941,
                        'tooth': None,
                        'type': 'metals'},
                    {   'bbox': [   800.822021484375,
                                    469.63043212890625,
                                    1654.4962158203125,
                                    637.3797607421875],
                        'class_name': 'ortodontia',
                        'contour': None,
                        'line': None,
                        'point': None,
                        'score': 0.831466555595398,
                        'tooth': None,
                        'type': 'metals'}],
    'height': 1292,
    'image_hash': None,
    'width': 2444}

Visualization¶

In [ ]:
from vis import draw_bboxes

dimage = draw_bboxes(panoramic_image, metals_response.json()["entities"])
fig, ax = plt.subplots(figsize=(10, 10))
ax.imshow(dimage)
ax.axis("off");
No description has been provided for this image

Teeth Segmentation Endpoint¶

Return a segmentation mask in the form of a contour for each found tooth on the panoramic image.

In [ ]:
teeth_segmentation_response = requests.post(
        BASE_URL + "/panoramics/teeth-segmentation",
        json={
            "base64_image": encode_image(panoramic_image),
        },
        headers=headers,
    )

Output Sample¶

In [ ]:
teeth_segmentation_response
Out[ ]:
<Response [200]>
In [ ]:
pdata = teeth_segmentation_response.json()
pdata["entities"] = [{**p, "contour": p["contour"][:3]} for p in pdata["entities"][:3]]
pp.pprint(pdata)
{   'entities': [   {   'bbox': None,
                        'class_name': '18',
                        'contour': [[608, 355], [607, 356], [606, 356]],
                        'line': None,
                        'point': None,
                        'score': None,
                        'tooth': None,
                        'type': 'teeth_segmentation'},
                    {   'bbox': None,
                        'class_name': '17',
                        'contour': [[701, 370], [700, 371], [699, 371]],
                        'line': None,
                        'point': None,
                        'score': None,
                        'tooth': None,
                        'type': 'teeth_segmentation'},
                    {   'bbox': None,
                        'class_name': '16',
                        'contour': [[828, 362], [827, 363], [826, 363]],
                        'line': None,
                        'point': None,
                        'score': None,
                        'tooth': None,
                        'type': 'teeth_segmentation'}],
    'height': 1292,
    'image_hash': None,
    'width': 2444}

Visualization¶

In [ ]:
from vis import contour2mask, draw_masks

contours_list = [[p["contour"]] for p in teeth_segmentation_response.json()["entities"]]
masks = [contour2mask(contour, *panoramic_image.size) for contour in contours_list]
dimage = draw_masks(panoramic_image, masks)
fig, ax = plt.subplots(figsize=(10, 10))
ax.imshow(dimage)
ax.axis("off");
No description has been provided for this image

Procedures Endpoint¶

Returns dental proceadures found on the panoramic image.

In [ ]:
procedures_response = requests.post(
        BASE_URL + "/panoramics/procedures",
        json={
            "base64_image": encode_image(panoramic_image),
        },
        headers=headers,
    )

Output Sample¶

In [ ]:
pdata = procedures_response.json()
pdata["entities"] = pdata["entities"][:1]
pp.pprint(pdata)
{   'entities': [   {   'bbox': None,
                        'class_name': 'MatRestCoron',
                        'contour': None,
                        'line': [   [632.8214285714287, 553.7142857142857],
                                    [611.0, 351.83928571428567]],
                        'point': None,
                        'score': 0.6624748706817627,
                        'tooth': '18',
                        'type': 'procedure'}],
    'height': 1292,
    'image_hash': None,
    'width': 2444}

Visualization¶

In [ ]:
from vis import draw_procedures_output

dimage = draw_procedures_output(panoramic_image, [e for e in procedures_response.json()["entities"] if e["score"] > 0.5])
fig, ax = plt.subplots(figsize=(12, 12))
ax.imshow(dimage)
ax.axis("off");
No description has been provided for this image

Describe Endpoint¶

The describe endpoint will return the result of all models at once.
The models can be called individually from the endpoints described in the docs.

In [ ]:
describe_response = requests.post(
        BASE_URL + "/panoramics/describe",
        json={
            "base64_image": encode_image(panoramic_image),
        },
        headers=headers,
    )

Output Sample¶

In [ ]:
pdata = describe_response.json()
pdata["entities"] = pdata["entities"][:3]
pp.pprint(pdata)
{   'entities': [   {   'bbox': None,
                        'class_name': 'normal',
                        'contour': None,
                        'line': None,
                        'point': None,
                        'score': 1.0,
                        'tooth': None,
                        'type': 'dentition'},
                    {   'bbox': None,
                        'class_name': 'superior_toothless',
                        'contour': None,
                        'line': None,
                        'point': None,
                        'score': 7.882013930426979e-10,
                        'tooth': None,
                        'type': 'dentition'},
                    {   'bbox': None,
                        'class_name': 'mixed',
                        'contour': None,
                        'line': None,
                        'point': None,
                        'score': 2.5856181778971532e-11,
                        'tooth': None,
                        'type': 'dentition'}],
    'height': 1292,
    'image_hash': None,
    'width': 2444}

Anomalies Heatmap Endpoint¶

Returns a heatmap of the anomalies found on the panoramic image.

In [ ]:
anomalies_response = requests.post(
        BASE_URL + "/panoramics/teeth-anomalies-heatmap",
        json={
            "base64_image": encode_image(panoramic_image),
        },
        headers=headers,
    )
In [ ]:
# Found classes:
for idx, data in enumerate(anomalies_response.json()):
    if idx > 7:
        break
    if len(data["anomalies"]) == 0:
        continue
    print(f"Tooth {data['tooth_name']}  Anomalies -" + ", ".join([anno["anomaly_name"] for anno in data["anomalies"]]), end=" ")
    print("\n")
Tooth 18  Anomalies -NodP 

Tooth 17  Anomalies -Calc, NodP, Car 

Tooth 16  Anomalies -Calc, NodP, Car 

Tooth 13  Anomalies -Calc 

Tooth 12  Anomalies -Calc, Car 

Tooth 11  Anomalies -Calc, Car 

Tooth 21  Anomalies -Calc, Car 

Visualization¶

Visualizing the Calc class.

In [ ]:
from utils import process_ret_api
from vis import draw_heatmap

heatmap = process_ret_api(anomalies_response.json(), h=panoramic_image.height, w=panoramic_image.width, anomaly2see="Calc")
dimage = draw_heatmap(panoramic_image, heatmap)
fig, ax = plt.subplots(figsize=(10, 10))
ax.imshow(dimage)
ax.axis("off");
No description has been provided for this image

Periapicals¶

In [ ]:
# The image is converted to grayscale in the backend, so you can convert it before sending in order to save bandwidth.
periapical_image = Image.open("images/periapical0.jpg").convert("L")

Classify Endpoint¶

The classify endpoint will return the periapical type among the 14 possible regions.

In [ ]:
classify_response = requests.post(
        BASE_URL + "/periapicals/classify",
        json={
            "base64_image": encode_image(periapical_image),
        },
        headers=headers,
    )

Output Sample¶

In [ ]:
pdata = classify_response.json()
pdata["entities"] = pdata["entities"][:4]
pp.pprint(pdata)
{   'entities': [   {   'bbox': None,
                        'class_name': '42-41-31-32',
                        'contour': None,
                        'line': None,
                        'point': None,
                        'score': 0.9996941089630127,
                        'tooth': None,
                        'type': 'periapical_classification'},
                    {   'bbox': None,
                        'class_name': '33',
                        'contour': None,
                        'line': None,
                        'point': None,
                        'score': 0.0002522027643863112,
                        'tooth': None,
                        'type': 'periapical_classification'},
                    {   'bbox': None,
                        'class_name': '43',
                        'contour': None,
                        'line': None,
                        'point': None,
                        'score': 3.3788826840464026e-05,
                        'tooth': None,
                        'type': 'periapical_classification'},
                    {   'bbox': None,
                        'class_name': '24-25',
                        'contour': None,
                        'line': None,
                        'point': None,
                        'score': 6.261640010052361e-06,
                        'tooth': None,
                        'type': 'periapical_classification'}],
    'height': 1621,
    'image_hash': None,
    'width': 1236}
In [ ]:
fig, ax = plt.subplots(figsize=(10, 10))
ax.imshow(periapical_image, cmap="gray")
ax.axis("off");
No description has been provided for this image

Longaxis Endpoit¶

Return the coordinates on the longaxis of each teeth found on the periapical image.

Sample Output¶

In [ ]:
longaxis_response = requests.post(
    BASE_URL + "/periapicals/longaxis",
    json={
        "base64_image": encode_image(periapical_image),
    },
    headers=headers,
)

pdata = longaxis_response.json()
pdata["entities"] = pdata["entities"][:3]
pp.pprint(pdata)
{   'entities': [   {   'bbox': None,
                        'class_name': '43_1',
                        'contour': None,
                        'line': None,
                        'point': [135.1875, 1215.75],
                        'score': 0.05661439150571823,
                        'tooth': None,
                        'type': 'longaxis'},
                    {   'bbox': None,
                        'class_name': '43_0',
                        'contour': None,
                        'line': None,
                        'point': [19.3125, 329.265625],
                        'score': 0.2382279932498932,
                        'tooth': None,
                        'type': 'longaxis'},
                    {   'bbox': None,
                        'class_name': '42_1',
                        'contour': None,
                        'line': None,
                        'point': [299.34375, 1063.78125],
                        'score': 0.45180168747901917,
                        'tooth': None,
                        'type': 'longaxis'}],
    'height': 1621,
    'image_hash': None,
    'width': 1236}

Visualization¶

In [ ]:
from vis import draw_longaxis_output

dimage = draw_longaxis_output(
    periapical_image, longaxis_response.json()["entities"], draw_axis=True, th=0.001
)
fig, ax = plt.subplots(figsize=(10, 10))
ax.imshow(dimage)
ax.axis("off");
No description has been provided for this image

Cephalometry¶

In [ ]:
tele_image = Image.open("images/tele0.jpg").convert("L")
In [ ]:
cefbot_response = requests.post(
    BASE_URL + "/cephalometry/points",
    json={
        "base64_image": encode_image(tele_image),
    },
    headers=headers,
)

pdata = cefbot_response.json()
pdata["entities"] = pdata["entities"][:3] 
pp.pprint(pdata)
{   'entities': [   {   'bbox': None,
                        'class_name': 'Nal',
                        'contour': None,
                        'line': None,
                        'model_name': 'cefalometry',
                        'point': [1399.0, 445.0],
                        'score': 0.875812828540802,
                        'tooth': None},
                    {   'bbox': None,
                        'class_name': 'Na',
                        'contour': None,
                        'line': None,
                        'model_name': 'cefalometry',
                        'point': [1330.0, 445.0],
                        'score': 0.9012552499771118,
                        'tooth': None},
                    {   'bbox': None,
                        'class_name': 'Or',
                        'contour': None,
                        'line': None,
                        'model_name': 'cefalometry',
                        'point': [1268.0, 765.0],
                        'score': 0.8242092132568359,
                        'tooth': None}],
    'image_hash': None,
    'model_name': 'cefalometry',
    'output_height': 2304,
    'output_width': 2104}

Detected Cepholometric points¶

Nal , Na , Or , SO , RO , Cl , S , Si , Sp , Cls , Po , Cli , Ba , Od , Bo , Op , Co , Bpc , Ar , Bac , C , Dc , Go , Me , Pog , Gn , E , B , Id , M , Pm , D , A , Ena , Pli , Enp , KR , Te , Cm , Ptm , PVT , Vasa , Vasp , Vaia , Vaip , Xi , Bar , D6s , Ams , 6s , Ppd , 6i , Ami , A4 , C4 , PAR , Ap , 3s , 3i , Aii , Iii , Iis , Sf1 , Ais , Pogl , Bl , Li , Stm , Ls , Al , Sn , Prn , Pn , EILI , AA , ADS , AD2 , R , AD1 , ADI , POR , D8 , M8 , D7 , V , T , Tuber , Pi , PTVR , LN , C2 , PPM , C2m , C2pi , C3ps , C3as , C3a , C3m , C3pi , C4ps , C4as , C4ai , C4m , C4pi

Visualization¶

In [ ]:
from vis import draw_points

dimage = draw_points(tele_image, cefbot_response.json()["entities"])
fig, ax = plt.subplots(figsize=(14, 14))
ax.imshow(dimage)
ax.axis("off");
No description has been provided for this image